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Abstract 



The relationships between course work taken, educational needs and plans, high 
school attended, and PLAN test scores of high school sophomores were examined. The 
relationships between ethnicity or gender and test scores, given students' course work 
taken, educational needs and plans, and high school attended, were then examined. The 
data consisted of two samples: a nationally-representative sample of 7,000 sophomores 
from 65 high schools, and a sample of 8,441 sophomores from 73 schools that tested all 
of their sophomores as part of the fall, 1992 PLAN operational administration. 

For both samples, course work taken, students' educational needs and plans, and 
high school attended were major factors in explaining students' achievement of higher- 
order thinking skills. Gender and ethnicity explained 2% or less of the variance in 
PLAN scores, over and above these factors. 
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Factors Influencing Differential Achievement of 
Higher-Order Thinking Skills, as Measured by PLAN 

The issue of differential performance on standardized tests has grown in 
importance in recent years. For example, higher-order thinking skills tests, such as those 
used for college admissions (ACT Assessment and SAT), have been closely scrutinized 
regarding differential performance by both ethnic and gender groups. Due to the "high 
stakes" nature of these assessments, it is important to idenkry those factors that make 
a difference in students' acquisition of higher-order thinking skills. 

The PLAN tests, while not "high stakes" per sc, are higher-order thinking skills 
tests that have implications for the academic futures of students. They are used both in 
educational planning for high school sophomores and in program and curriculum 
evaluation. Because PLAN mean scores have been shown to differ across ethnic and 
gender groups (Noble, 1991), the investigation of factors influencing differential test 
performance is important. 

Research on differential performance on standardized tests (largely by ethnicity 
and /or gender) has showed associations between high school grades, course work taken, 
student and high school characteristics, students' educational needs and plans, and test 
performance (e.g., Noble & McNabb, 1989; Chambers, 1988; Pallas & Alexander, 1983). 
Noble, Crouse, Sawyer and Gillespie (1992), for example, found that high school course 
work and course grades were strong predictors of ACT Assessment performance. 
Additionally, past research has shown a reduction in the role played by ethnic and 
gender variables in accounting for variation in standardized test scores when course 
work taken and course grades earned were statistically controlled (Noble, et ah, 1992; 
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Noble & McNabb, 1989; Pallas & Alexander, 1983). The use of course work and course 
performance along with background and educational needs and plans variables, and 
high school attended as control variables resulted in a decrease in the variation in ACT 
Assessment scores attributable to ethnicity. The percent of ACT score variance 
accounted for by either ethnicity or gender, when statistically controlling for the other 
variables, was only 1% (Noble, et al., 1992). 

Research studies on PLAN scores have largely been descriptive in nature. 
Previous research has found that sophomores who took, or were currently taking, 
specific kinds of courses achieved, on average, higher PLAN scores than students who 
did not take the courses (Noble, 1991). African-American and Hispanic (Mexican- 
American, Chicano, Puerto-Rican, Cuban, other Hispanic origin) students who were 
currently taking or planning to take college-preparatory course work had higher PLAN 
scores than students not taking the course work (Noble, 1991). In a study of PLAN 
performance in one state. Noble (1990) found substantial differences in the average 
PLAN scores of students taking and not taking college-preparatory core courses. Mean 
gender differences typically favored females for all PLAN tests; differences by ethnicity 
typically favored Caucasian-Americans and Asian-Americans (ACT, 1994). 

In the present study, the impact of course work taken, students' educational needs 
and plans, and high school attended was considered in explaining variation in PLAN 
scores of high school sophomores. (Course grades could not be obtained for this study). 
The relationships between test scores and ethnic and gender group membership were 
then considered by statistically controlling for course work taken, students' educational 
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needs and plans, and high school attended. In contrast to previous PLAN research, this 
study used multiple regression and analysis of covariance techniques to examine the 
aggregate effects of these variables on PLAN performance. 

Including high school attended in the study helps to determine the overall impact 
of high school attended on PLAN score performance. Moreover, it helps in identifying 
differences among schools in the relationships between course- taking, educational needs 
and plans, and students' acquisition of higher-order thinking skills. Further, identifying 
the characteristics of higher- or lower-scoring schools helps in understanding the context 
in which students learn (or do not learn) these skills. 

The population of students completing PLAN each year differs in several ways 
from high school sophomores in general. Typically, PLAN-tested students are likely to 
be more academically able than typical sophomores, and are more likely to attend 
private and smaller high schools from the North Central and Southern accrediting 
regions. Of schools that operationally administer PLAN in a given year, about 35% test 
all of their sophomores; 65% of the schools test a more selective group of high-achieving, 
college-bound students. As such, the result? based on a nationally-representative sample 
of sophomores may differ from those obtained using the PLAN-tested sophomore 
population. Therefoie, the independent variables identified as important for the 
nationally-representative sample of students were used to estimate PLAN scores for a 
second group of sophomores who completed PLAN in fall, 1992. This group consisted 
of a representative sample of students from schools that tested all of their sophomores 
(denoted as the "all-sophomon te: led" sample). The results for both groups of 
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sophomores were then compared. Analyses were also conducted on data from schools 
testing a more selective group of PLAN-tested sophomores; the results are not reported 
here but may be obtained from the first author. 

Data 

A stratified random sample of approximately 65 high schools with 7,000 students 
participated in a fall, 1992 national norming study for PLAN. The schools were stratified 
by affiliation (public/private), ACT user status (user=25 or more ACT-tested students 
each year) and by estimated enrollment in Grade 10, resulting in 10 strata. The average 
obtained student response rate within schools was 92%. 

The 1992 PLAN-tested population consisted of 372,037 sophomores from 4,371 
schools, each testing a minimum of 25 students. Of the total, 197,587 sophomores were 
from all-sophomore tested schools (1,719 schools). These schools were categorized, 
stratified, and proportionately sampled based on the number of students tested and 
average PLAN Composite score. Average PLAN Composite score was used to insure 
representation of a broad range of educational achievement across schools. The resulting 
sample consisted of sophomores from 73 all-sophomore tested schools (8,441 students). 

The variables for the study included five PLAN test scores representing the four 
academic tests (English, Mathematics, Reading and Science Reasoning) and the 
Composite. The Composite score is the arithmetic mean of the four test scores, rounded 
to the nearest whole number. 
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Course Work Taken 

Course work taken (previously taken or currently taking) and planned was 
obtained from the Course Information Section of PLAN, which collects self-reported 
information about 30 high school courses. Research has shown that PLAN-tested 
sophomores provide accurate self-reports of course work they have taken or are 
currently taking (ACT, 1995). Course work taken combinations were quantified by a 
series of dichotomous variables indicating whether or not the students had taken or was 
currently taking the course work. The particular course work combinations were based 
on prior PLAN research (Noble, 1990; Noble, 1991) and included: 

1. English 9 or 10 (taking one or both courses vs. taking neither course). 

2. English 9 and 10, or English 9 and Speech (taking vs. not taking both 
courses in either pair). 

3. Algebra 1 and Algebra 2, or Algebra 1 and Geometry (taking vs. not taking 
both courses in either pair). 

4. Algebra 1, Algebra 2, or Geometry (taking one or more of these courses vs. 
not taking any of the courses). 

5. Art, Music or Drama/Theater (taking one or more of these courses vs. not 
taking any of the courses). 

6. Foreign Language (taking one or more foreign languages vs. not taking any 
foreign language). 

7. Foundational Course Work (taking English 10, Algebra 1 and any other 
Mathematics course, any Social Studies course, and Biology vs. not taking 
one or more of these courses). 

8. English 9 and 10, or English 9 and Speech; Algebra 1 and Algebra 2, or 
Algebra 1 and Geometry; any Social Studies course; and General Science 
and Biology (taking all of these courses vs. not taking one or more of the 
courses). 
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Educational Needs and Plans 

Students' planned course work was determined by students' self-reports of their 
plans to complete clusters (combinations) of particular courses. A total of 21 clusters 
were defined, based on prior research (Noble & McNabb, 1989); see Appendix A for the 
definitions of particular clusters. Each cluster of courses was represented by dummy- 
coded variables, indicating whether or not the student planned to take all of the courses 
in a particular cluster. 

Indices of a student's educational needs and plans also included his/her 
educational plans (plans to attend a two-year or four-year postsecondary institution) and 
self-reported needs for help (in Reading, Writing, Mathematics, Study Skills, and Test 
Taking). The self-reported needs for help are given on an ordinal scale, i.e., "a lot", 
"some", "little/none", with values ranging from 0 (little/none) to 2 (a lot). 

High School Attended 

High school attended was entered as an effect-coded dummy variable for each 
high school. School characteristics were obtained from a file maintained by Market Data 
Retrieval in Shelton, Connecticut. The characteristics included school affiliation (public, 
private), accrediting region, location (rural, suburban, urban), total per-pupil 
expenditure, percent below federal poverty level in the district, percentages of African- 
Americans and Hispanics in the district, and school enrollment. 

Gender and Ethnicity 

Students' gender and ethnicity were also included as dummy variables. Due to 
small sample sizes among several ethnic groups (e.g., Asian-Americans), separate >ffect 
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estimates were derived only for African-Americans, Caucasian-Americans, Hispanics 
(Mexican-American, Chicano, Puerto Rican, Cuban, other Hispanic origin), and students 
of other ethnic origins. For gender, males were coded as 0 and females as 1; for 
ethnicity, dummy variables were coded such that African-Americans, Hispanics, and 
students of other ethnic origins were compared to Caucasian-Americans. 

Method 

Weighting was done with the national sample to more closely approximate the 
performance of sophomores nationwide. Sampling weights were calculated as follows: 

W = c*wl*w2, (1) 

where c = a constant, 

wl = (total number of schools in a stratum) /(total number of schools in 
the sample associated with a stratum), 
w2 = (estimated enrollment in grade 10)/ (total number of students tested 
within a school on a given test form). 

Results for the national sample were weighted according to W. The all-sophomore 
sample results were not weighted. 

Characteristics of the schools and students in each sample were first summarized; 
means and standard deviations for all variables were computed for each sample, and by 
ethnic and gender group within each sample. Simple correlations were then calculated 
between all independent and dependent variables for both samples. 

Multiple regression models were developed for predicting PLAN scores using the 
national sample data. The following sets of independent variables were used and 



8 



entered in this order: (1) courses taken, (2) educational needs and plans variables, (3) 
high school attended, and (4) ethnicity or gender. Considerations in model development 
included the statistical significance of simple correlations between prospective 
independent variables and PLAN scores (p < .001 for all variables except ethnicity and 
gender, where p < .05), model parsimony, and collinearity among variables. Individual 
variables were selected for each variable set in sequential order; e. ., variables were 
selected to represent courses taken, and then variables were selected to represent 
students' educational needs and plans, conditioned upon those representing courses 
taken. A condition number of 15 or greater (with two or more variance-decomposition 
proportions exceeding .50) was regarded as indicative of collinearity (Belsley, Kuh, & 
Welsch, 1980). The final models were then used to estimate PLAN scores for the all- 
sophomore tested sample. Adjusted means were computed for each ethnic and gender 
group, by sample. All regression results were then compared for the two samples. 
Effects of Ethnicity, Gender, and High School Attended 

Based on the regression results, a second set of analyses was conducted using 
both samples to determine whether the relationships between course work taken or 
planned and PLAN scores were moderated by students' ethnicity, gender, or high school 
attended. Interaction terms were included in the regression models for these effects; the 
course work (taken and planned) variables included in the interactions were the primary 
course work variables in the final regression models. Statistically significant (p < .05) 
interactions between ethnicity or gender and course work taken and planned were 
plotted to determine the nature of the interactions. 
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For statistically significant interactions involving high school attended, within- 
school regression models were developed, and the distributions of predicted PLAN 
means across high schools were -tudied. High schools with predicted mean PLAN 
Composite scores that differed by ±1 or more from the pooled predicted mean PLAN 
Composite across all students and schools were identified. Predicted school mean 
differences were also required to be statistically significant (p < .001). (Note: Significance 
tests were based on individual school error variances, rather than pooled error 
variances.) The regression weights for course work taken and planned, and the 
characteristics of low-scoring (i.e., "below expected"; predicted mean difference of -1 or 
more) and high-scoring ("above expected"; predicted mean difference of +1 or more) 
schools were then compared. This approach, rather than categorizing schools on mean 
PLAN Composite score, would take into consideration differences in the relationship 
between course work and PLAN performance across high schools; i.e., lower-scoring 
schools are those scoring below what would typically be "expected," given their students' 
course work, educational needs and plans, and ethnicity or gender. 

Results 

Descriptive Statistics 

School characteristics. Table 1 provides a comparison of the schools comprising the 
national and all-sophomore tested samples. The all-sophomore sample consisted of 
more private schools (34% vs. 23%), more schools from the North Central accrediting 
region (68% vs. 52%), and more schools with higher per-pupil expenditure (29% vs. 52% 
with expenditures of $4200 or higher) than the national sample. Further, the schools in 
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TABLE 1 

School Characteristics of National and All-sophomore Tested PLAN Samples 

(Percentages) 



Characteristic 


Category 


National 
(64 schools) 


All-sophomore tested 
(72 schools) 


Affiliation 


Public 


77 


66 




Private 


23 


34 


Accrediting region 


South 


33 


24 




West 


0 


3 




Middle 


8 


6 




North Central 


52 


68 




Northwest 


5 


0 




Northeast 


3 


0 


Location 


Urban 


13 


26 




Suburban 


22 


36 




Rural 


65 


38 


Per pupil expenditure* 


< $2200 


0 


2 


S2200-S3199 


36 


18 




S3200-S4199 


35 


28 




$4200-55199 


15 


32 




$5200 and over 


14 


20 


Percent below federal 


0-4.0% 


13 


20 


poverty level in the district* 


5-11.9% 


26 


40 




12-24.9% 


34 


32 




25° o and over 


28 


8 


% African- Ain. students in 


0% 


44 


50 


the district 


1-24% 


53 


46 




25% and over 


3 


4 


% Hispanic students in the 


0% 


32 


31 


district 


1-24% 


68 


59 




25% and over 


0 


10 


School enrollment 


1-99 


1 


1 




100-299 


25 


24 




300-499 


28 


28 




5(X)-999 


30 


25 




1000 and over 


17 


22 



* Public schools onlv 
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this sample had fewer African-Americans (50% vs. 44% with no African-American 
students), and were less likely to be in districts with higher poverty rates than the 
national sample (40% vs. 62% with 12% or more below the federal poverty level). 

PLAN scores. Means, standard deviations, and sample sizes for PLAN scores are 
reported in Table 2. Statistics are reported for each sample by ethnicity, gender, and the 
total group. Of those students in the national sample indicating their ethnic affiliation 
(n=5,972), approximately 73% were Caucasian-American, 11% were African-American, 
7% were Hispanic, and 9% were of other ethnic origin. Of the students in the all- 
sophomore tested sample who reported their ethnicity (n=7,252), 6% were African- 
American, 77% were Caucasian-American, 8% were Hispanic, and 9% were other. 

Mean PLAN scores for the national sample were generally lower than those for 
the all-sophomore tested group of students by about 1.5 to 2.0 scale score units (about 
two-fifths of one standard deviation). Further, for all PLAN tests except English, 
standard deviations tended to be smaller for the national sample than for the all- 
sophomore tested sample. 

With the exception of PLAN Mathematics, means for females in the national 
sample were slightly higher than those for males, with the differences ranging from 1.6 
PLAN score units on the English test to .29 PLAN score units on the Science Reasoning 
test. On the Mathematics test, males and females scored similarly. For the all- 
sophomore sample, males and females score similarly on all tests except English and 
Mathematics. Females scored higher than males on the English test (by 1.0 PLAN score 
units; about one-third of one standard deviation) and slightly lower than males on the 



PLAN Score Means and Standard Deviations, by Sample 



JO 



PLAN score 


Composite 


SD 


National 


3.78 


3.87 


3.64 


2.69 


3.73 


3.31 


3.23 




Tb 

O' 

CO 


4.06 


3.78 


3.29 


LA 

X 

CO 


3.24 


3.80 


Mean 


16.3 


16.0 


16.7 


14.1 


17.4 


00 


15.0 




18.1 


18.0 


18.2 


00 
l a 


18.8 


16.1 


17.2 


to 

c 

d 

o 

(A 

rc 

Q> 


SD 


3.41 


3.56 


3.24 


2.82 


3.38 


O' 

T— i 

CO 


3.13 




3.65 


l 

3.89 


s X 

CO 

CO 


3.10 


04 

dO 

cd 


3.04 


CO 

cd 


Cd 

Q> 

u 

d 

Q> 

*u 

CD 


Mean 


16.5 


x 


16.7 


OC 


17.3 


vO 
l ri 


15.8 


18.0 


18.0 


X 


16.1 


18.6 


LA 


17.4 


Reading 


SD 


00 

-b 


4.78 


4.71 


3.53 


4.79 


4.47 


4.01 




n 

4.91 


oc 

LA 


4.72 


O' 


-t* 

OC 


LA 


tX 

-r 


Mean 


LA 


15.2 


16.3 


CO 

CO 


17.1 


14.3 


O' 

cc 


17.7 


vC 

id 


17.8 


lA 


LA 

: d 


nC 

LT 


16.5 


Mathematics 


SD 


o 

o 


4.26 


3.72 


3.07 


3.99 


3.45 


3.74 


1 

4.22 


4.30 


4.11 


-V 
l n 

CA 


4.15 


CO 

cd 


4.35 


Mean 


cO 

x 


vC 

T— < 


CO 

r h 


14.5 


17.3 


X 

r— < 


15.6 


17.9 


(N 

OC 


17.7 


-r 

in 


oo 


cr 

la' 


id 


English 


SD 


4.96 


t— < 

O' 


4.89 


o 

00 

CO 


O' 

CO 


| 4.31 


4.34 


4.98 


5.00 


O' 


4.20 


4.87 


PO 

CN 

—b 


4.85 


Mean 


16.1 


15.3 


o 

o 


13.5 


17.6 


(N 

rb 


14.3 


18.2 


17.7 


18.7 


X 

la 


19.1 


X 

LA 


0 s 




i 


Sample 

size 


6980 


3512 


3463 


645 


4359 


r 

Ob 


537 


All-sophomore tested 


8441 


4491 


X 

CO 


474 


5582 


X 

LT. 


X 

-*■ 


Group 


Total 


Males 


Females 


Afr.-Ams. 


Cau.-Ams. 


Hispanics 


Other 


Total 


Males 


<S‘ 

~rt 

E 

o 

U- 


Afr.-Ams. 


Cau.-Ams. 


Hispanics 


Other 
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Mathematics test (by .5 PLAN score units; less than one-tenth of one standard deviation). 

Caucasian-American students consistently outscored the other ethnic groups in 
both samples, with the greatest differences on the PLAN English test and the smallest 
differences on the PLAN Science Reasoning test. Caucasian-American students' scores 
typically exceeded those of Hispanics by about 2.5 PLAN score units for all PLAN tests. 
Scores for Caucasian-American students typically exceeded those of African-American 
students by about 3.2 PLAN score units. Mean score differences between Caucasian- 
Americans and African-Americans, and between Caucasian-Americans and students 
indicating "other ethnic origin," were similar for the two samples. 

Course toork taken. Descriptive statistics for the various course cluster variables, 
indicating whether students had taken or were taking particular groups of courses are 
reported in Table B-l (Appendix B) for both samples. The table includes the percentages 
of students completing the various clusters. The correlations of the cluster variables with 
PLAN scores are also shown. The course work taken variable that was most strongly 
associated with all five PLAN scores was whether or not the student had taken Algebra 
1 and Algebra 2, or Algebra 1 and Geometry, with correlations ranging from .37 to .54 
across both samples. A higher percentage of students in the all-sophomore sample than 
the national sample had taken or were taking these courses. Whether or not the student 
had taken any foreign language (correlations= .25 to .34), or had taken the foundational 
course work (r = .35 to .47) was also positively associated with PLAN scores. 

Need variables. Table B-2 (Appendix B) contains means and standard deviations 
for the various need variables, along with the correlations of these variables with PLAN 
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scores. Correlations were fairly consistent across the two samples; greater needs for help 
were associated with lower test scores. Needs for help in developing test taking skills 
correlated consistently with all of the PLAN scores (r = -.25 to -.35). Needs for help in 
developing writing skills, increasing reading speed, and increasing understanding of 
what is read were most strongly related to PLAN English, Reading and Composite 
scores. Needs for help in developing math skills correlated most highly with PLAN 
Mathematics scores (r = -.35 and -.38). Needs for help in developing speaking skills and 
the needs for help in choosing a college and choosing a job were minimally associated 
with PLAN scores. 

Planned course zoork. Descriptive statistics for the clusters of courses that students 

planned to take are reported in Table B-3 (Appendix B). The table includes the 

percentages of students planning to complete the various clusters, and correlations with 

PLAN scores. The percentages of students in both samples planning to take the courses 

were similar for all courses except social studies courses, where a larger percentage of 

national students than all-sophomore tested students were planning to take these 
* 

courses. 

In general, planned course work in mathematics (r = .23 to .37) and in natural 
sciences (r = .13 to .26) was more strongly associated with PLAN scores than other 
planned course work variables. This finding was consistent across both samples. Unlike 
course work taken, however, planned course work in foreign languages typically 
correlated negatively with PLAN score performance (r = -.01 to -.10). 
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Regression Analyses 

Results of the icgre sicn analyses are reported in Tables 3 through 6: Tables 3 
(gender) and 4 (ethnicity) contain the results for the national sample and Tables 5 
(gender) and 6 (ethnicity) contain the all-sophomore results. Each table shows the 
breakdown in the contributions of the variable groups to explairing variability in PLAN 
scores. The tables include the regression weight for each variable and the R 2 
contribution for each variable group. The tables also show the overall R 2 and SEE 
(standard error of estimate) for each regression analysis. 

Course work taken. The primary course work taken variables included in the final 
models were indicators of whether or not the student had taken either Algebra 1 & 
Algebra 2 or Algebra 1 & Geometry; any foreign languages; and for PLAN English, 
course work taken in Art, Music, or Drama. All other variables either did not contribute 
significantly to the model and/or were collinear with these variables. 

Across PLAN tests and samples, course work taken explained 17% to 31".;, of the 
total variance in PLAN scores; in general, R 2 values were slightly higher for the all- 
sophomore tested sample than for the national sample. Algebra 1 and Algebra 2, or 
Algebra 2 and Geometry course work taken was more strongly associated with PLAN 
scores for the all-sophomore tested group than for the national sample. The variance 
accounted for by course work taken was generally lowest for Reading and Science 
Reasoning and highest for PLAN Mathematics and the Composite. 
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Educational need* and plans. The educational needs and plans variables most 
effective in explaining PLAN scores were the measures of student need and course work 
planned in mathematics and foreign languages. The particular need variables varied 
from test to test, however. Needs for help in developing test taking skills was 
consistently associated with PLAN scores, over and above course work taken (regression 
weights = -.65 to -1.27); the regression weights for help in test taking were higher for the 
all-sophomore sample than for the national sample, however. Other important need 
variables were needs for help in developing mathematics skills and in increasing reading 
speed. 

Planned course work. Over and above course work taken, the indicators of whether 
or not students planned to take Algebra II, Geometry, Trigonometry, and Calculus, or 
j lanced to take both Spanish and French were the primary course work planned 
variables for explaining PLAN scores. Consistent with course work taken, planned 
mathematics course work was more strongly associated with PLAN scores for the all- 
sophomore tested sample than for the national sample (regression weights = .99-1.51 vs. 
.55-1 .08). The need variables and the planned course work variables together accounted 
for between 5% and 9% of the total variance in PLAN scores, over and above course 
work taken for the national sample. For the all-sophomore sample, these variables 
accounted for between 8'T and 12°<> of additional variance in PLAN scores. 

Iliph school attended. The two samples also differed in the contribution of high 
school attended, over and above course work taken and educational needs and plans. 
I oi the national sample, high school attended accounted for between IP’,, and 14"i> of 
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additional variance, compared to 6% to 8% for the all-sophomore tested sample. 

Gender and ethnicity. Gender or ethnicity consistently accounted for a minimal 
proportion of additional variance in PLAN scores. Gender explained, at n ;st, T'l. of 
PLAN score variance, over and above the other variables in the models. For the PLAN 
Mathematics test, the increase in R 2 by gender was not found to be statistically 
significant (p > .05), over and above the other variables, for either sample. The 
percentage of additional PLAN score variance explained by ethnicity ranged from 1% 
to 2%, over and above the other variables in the models. 

Compared to other PLAN scores, the greatest amount of variability was explained 
for PLAN Mathematics and PLAN Composite scores for both samples (R 2 = .47 to .51). 
The smallest SEE values were found for the Composite for both samples (2.63 to 2.80). 

Adjusted mean differences. Tables 7 and 8 provide unadjusted and adjusted means 
for each gender and ethnic group, by sample. The adjusted mean represents an average 
PLAN score for a group, controlling for all independent variables other than the 
grouping variable itself. The adjusted and unadjusted means were rescaled by setting 
the means for males and Caucasian-Americans equal to the total group mean. The 
corresponding means for females and the nonwhite ethnic groups were computed 
relative to the means for males and Caucasian-American students. 

Adjusted mean differences between males and females from the national sample 
(when controlling for courses taken, educational needs and plans, and high school 
attended) typically were slightly smaller (20-29°;. smaller) than the corresponding 
unadjusted mean dillereiu.es, except lor PI AN Mathematics anil t L'ienie Reasoning (mv 



Table* 7). For these tests, the adjusted mean differences were the same as the unadjusted 
mean differences. 

For the all— sophomore tested sample, the adjusted mean gender differences were 
similar to or slightly larger than the unadjusted mean differences, except for 
Mathematics, where the adjusted mean differences were slightly smaller. The only 
adjusted mean differences that appeared to favor males from both samples were for 
Mathematics; these differences were not statistically significant, however. 
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Unadjusted and Adjusted PLAN Means by Gender and Sample 



Test 


Mean difference 
(females-males) 


Unadjusted mean 


Adjusted mean 


Unadjusted 


. djusted 
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Females 


Males 


Females 


National 


Fn^lish 


1.6 


1.2 
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16.1 
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1 0 
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Unadjusted and Adjusted PLAN Means by Ethnicity and Sample 
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Mean differences between African-American and Caucasian-American students 
were reduced by 49 to 54% by controlling for course work taken, educational needs and 
plans, and high school attended. The largest reduction was on the PLAN Mathematics 
test, and the smallest reduction was on the PLAN Science Reasoning test. Mean 
differences between Hispanic and Caucasian-American students were reduced by 25 to 
46%, with the largest reduction on the PLAN English test and the smallest reduction on 
the PLAN Science Reasoning test. 

Effects of Ethnicity, Gender, and High School Attended 

Ethnic group by mathematics course work taken (i.e. Algebra 1 and Algebra 2, or 
Algebra 2 and Geometry) and by mathematics course work planned (i.e.. Algebra 2, 
Geometry, Trigonometry, and Calculus) interaction terms were added to the regression 
models. The results showed statistically significant (p < .05) ethnic group by 
mathematics course work planned interactions for the PLAN Composite for the all- 
sophomore sample, and for PLAN Science Reasoning for the national sample. In both 
cases the mean PLAN score differences between students planning to take and not 
planning to take the course work were greater for Caucasian-American students than for 
students from other ethnic groups, as shown in Figure 1. Though statistically significant, 
the mean score differences suggest that the interactions were not practically significant. 
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FIGURE 1. Ethnicity by Planned Mathematics Course Work Interaction-PLAN Composite 



Statistically significant (p < .001) mathematics course work taken and course work 
planned by high school attended interactions were also found. For the all-sophomore 
sample, significant high school by course work planned interactions were found for all 
PLAN tests except Reading; the only significant course work taken by high school 
interaction occurred for PLAN Mathematics. For the national sample, statistic lly 
significant high school by mathematics course work taken interactions were found for 
all PLAN tests. 

Within-school regression models were developed for both the national and all- 
sophomore samples. Due to small sample sizes, the numbers of high schools were 
i educed to 60 and 65, respectively. The regression weights associated with mathematics 
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course work taken and course work planned were then summarized across institutions, 
as shown in Table 9. 

The regression weights for both mathematics course work taken and planned 
varied widely across high schools. Negative weights associated with these variables 
were found for some high schools; median values, however, were consistently positive 
across PLAN tests and samples. As was found earlier in this study using the high 
school dummy variable regression models, the regression weights associated with 
mathematics course work taken and planned were larger for the all-sophomore sample 
than for the national sample. 

High school groups. Three general categories of high school were identified for 
each sample. Schools with predicted mean PLAN Composite scores within ±1 (one 
standard error of measurement) of the pooled predicted mean were identified as 
performing "as expected," given their students' course work, educational needs, and 
plans, and ethnicity. Those scoring below the pooled predicted mean Composite score 
were identified as performing "below expected"; schools scoring above the pooled 
predicted mean Composite were identified as performing "above expected." The 
regression weights for mathematics course work taken and planned were then 
summarized across the schools in each school group, by sample, as shown in Table 10. 
The number of schools in each group are reported in parentheses in the shaded rows. 
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TABLE 9 

Regression Weights for Mathematics Course Work Taken and Planned — 
National and All-sophomore Tested Samples 



PLAN test 


Taken Alg. 1 & Alg. 2, or 
Alg. 1 & Geometry 


Plan to take Alg. 2, Geom., 
Trig., & Calc. 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


National (60 schools) 


English 


2.65 


-4.68 


6.64 


.93 


-2.36 


8.96 


Mathematics 


2.83 


-4.19 


5.83 


1.31 


-3.08 


4.59 


Reading 


2.30 


-5.16 


6.34 


.91 


- 3.93 


5.57 


Science Reasoning 


1.67 


-4.22 


7.92 


.73 


2.56 


3.78 


Composite 


2.49 


-2.84 


6.67 


1.09 


-1.58 


5.90 


All-sophomore tested (65 schools) 


English 


3.02 


.31 


7.90 


1.04 


-4.78 


5.48 


Mathematics 


3.02 


.19 


7.37 


1.46 


-2.01 


4.93 


Reading 


3.09 


-2.19 


8.70 


1.06 


-1.97 


4.79 


Science Reasoning 


2.28 


-1.07 


9.00 


1.03 


-2.44 


6.50 


Composite 


2.78 


.29 


5.91 


1.11 


-2.20 


3.91 



For the national sample, for which the high school by mathematics course work 
taken interaction was found, the regression weights associated with mathematics course 
work taken were generally smaller for the below expected group; i.e., the relationship 
between mathematics course work taken and PLAN performance was weaker for schools 
in this group than for those in the other groups. For the all-sophomore sample, a 
similar trend was found for mathematics course work planned: The relationship 

between planned mathematics course work and PLAN performance was generally 
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weaker for schools in the below expected group than for schools in the other two 
groups. 

As shown in Table 11, schools in the below expected group typically were large, 
public, rural schools f: om the Southern accrediting region. These schools were more 
likely to have lower total per-pupil expenditures, higher percentage;, of families with 
incomes below federal poverty level, and higher percentages of African-American 
students in the district than schools in the other two groups. In contrast, schools in the 
above expected group were typically suburban, public and private schools from the 
North Central accrediting region. These schools had higher total per-pupil expenditures, 
fewer families with incomes below the federal poverty level, and fewer African- 
American and Hispanic students than schools in the other two groups. It should be 
noted, however, that large, public, rural schools with low per-pupil expenditures, higher 
percentages of families with incomes below' the federal poverty level, and higher 
percentages of African-Americans or Hispanics could also be found in the as expected 
and above expected groups. 
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TABLE 10 



Regression Weights for Mathematics Course Work Taken and Planned — 
National and All-sophomore Tested Samples, by School Group 





Taken A 


Ig. 1 & Alg. 2, or Alg. 1 & Geometry 


Pian to take Alg. 2, Geom., Trig., & Calc. 




Bel 


ow expec 


:ted 


As expected 


Above expected 


Below expected 


As expected 


Above expected 


PLAN test 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


National 




(8) 






(39) 






(13) 






















English 


1.93 


-.19 


5.66 


2.65 


-.88 


6.64 


2.99 


-4.68 


5.36 


1.10 


.31 


4.00 


1.00 


-1.83 


8.96 


.71 


-.43 


4.45 


Mathematics 


1.87 


-.51 


5.83 


2.97 


-.98 


5.52 


2.33 


-4.19 


4.15 


1.15 


.29 


2.12 


1.35 


-1.52 


4.59 


1.74 


.09 


3.95 


Reading 


.82 


-1.21 


4.C7 


2.38 


-3.11 


6.34 


3.60 


-5.16 


5.02 


1.78 


.35 


3.92 


.59 


-3.93 


5.57 


1.76 


-1.34 


4.76 


Science Reasoning 


1.24 


-.65 


4.15 


1.82 


-4.23 


5.18 


1.75 


-1.28 


7.92 


.70 


.21 


1.22 


.58 


-2.56 


3.78 


1.14 


.26 


2.53 


Composite 


1.04 


-.26 


5.01 


2.57 


-.54 


6.67 


2.64 


-2.84 


4.19 


1.34 


.43 


2.95 


.94 


-1.58 


5.90 


1.51 


-.64 


4.47 


Ail-sophomore tested 




(ii) 






(45) 






0) 






















English 


2.94 


.42 


6.14 


3.02 


.31 


7.90 


3.53 


1.35 


4.89 


.28 


-2.81 


5.48 


1,18 


-4.78 


4.11 


1.39 


.52 


3.48 


Mathematics 


2.53 


.19 


4.82 


3.08 


.88 


7.37 


3.47 


.94 


4.75 


-.23 


-2.01 


2.14 


1.75 


-1.56 


4.65 


1.47 


.74 


4.93 


Reading 


2.54 


.54 


6.21 


3.40 


-2.19 


8.70 


1.99 


-.24 


8.46 


.25 


-1.37 


4.02 


1.06 


-1.97 


4.79 


1.33 


.46 


4.28 


Science Reasoning 


2.09 


.93 


4.27 


2.23 


-.28 


9.00 


2.62 


-1.07 


3.77 


.39 


-1.18 


2.06 


1.07 


-2.44 


6.50 


1.20 


.86 


2.44 


Composite 


2.89 


.29 


5.43 


2.78 


.83 


5.91 


1.94 


.92 


4.22 


.42 


-1.40 


3.91 


1.21 


-2.20 


3.21 


1.11 


.53 


2.74 
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TABLE 11 

School Characteristics of National and All-sophomore Tested PLAN Samples by School Group 

(Percentages) 







National 


All-sophomore tested 






Below 


As 


^ 

1 Above 


Below 


As 


Above 


Characteristic 


Category 


expected 

'i) 


expected 

(39) 


j expected 
J (13) 


expected 

(ID 


expected 

(45) 


expected 

(9) 


Affiliation 


Public 


100 


82 


i 69 


90 


57 


89 




Private 


0 


18 


j 31 


10 


43 


11 


Accrediting region 


South 


50 


33 


i 8 


54 


18 


11 




W>st 


0 


0 


i 0 


9 


2 


0 




Middle 


13 


8 


1 

* 7 

i 


0 


2 


22 




North Central 


38 


57 


J 69 


36 


78 


67 




Northwest 


0 


3 


1 8 


0 


0 


0 




Northeast 


0 


0 


i 8 


0 


0 


0 


Location 


Urban 


0 


13 


1 17 


27 


24 


33 




Suburban 


25 


15 


j .2 


27 


31 


56 




Rural 


75 


72 


i 42 


46 


41 


11 


Per pupil expenditure* 


< $2200 


0 


0 


0 


10 


0 


0 




$2200-53199 


50 


47 


t 

1 22 


30 


21 


0 




S3200-S4199 


25 


38 


s 44 


20 


28 


25 




$4200-55199 


13 


9 


i 11 


20 


28. 


50 




$5200 and over 


13 


6 


j 22 


20 


17 


25 


Percent below federal 


0-4.9% 


0 


6 


' 33 


0 


10 


75 


poverty level* 


5-11.9% 


13 


13 




30 


52 


13 




12-24.9% 


13 


56 


1 

i 0 


60 


28 


13 




25% and over 


75 


25 


; 0 


10 


10 


0 


% African-American 


0 % 


13 


46 


I 62 


46 


51 


44 


students 


1-24% 


63 


49 


i 39 


45 


45 


56 




25% and over 


25 


5 


I - 


9 


4 


0 


% Hispanic students 


0 % 


25 


28 


j 54 


18 


33 


11 




1-24% 


75 


64 


| 46 


55 


58 


89 




25% and over 


0 


8 


i ° 


27 


9 


0 


School enrollment 


1-99 


0 


3 


-4 0 


0 


2 


0 




100-299 


0 


23 


i 23 


0 


33 


0 




300-499 


13 


28 


1 

! 39 


36 


29 


33 




500-999 


63 


28 


i 31 


36 


20 


11 




1000 and over 


25 


18 


1 8 
i 


27 


16 


56 



Public schools only 
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Discussion 

The study confirmed past research findings on the role of courses taken or being 
taken as a major factor in explaining students' achievement of higher-order thinking 
skills, as measured by PLAN. The role of educational needs and plans was also 
assessed, with the results supporting the power of these variables, over and above 
course work taken, in explaining PLAN score variance. Further, the results supported 
the role of high school attended as a strong predictor of students' higher order thinking 
skills. 

Gender provided little explanatory power, over and above the other variables in 
the models. The study clearly supports the conclusion that gender is minimally related 
to levels of higher-order thinking skill, as measured by PLAN scores, over and above 
courses taken, educational needs and plans, and high school attended. These results 
support the findings of past research, (e.g., Noble, et al., 1992) on the role of gender in 
the explanation of score variance on other, higher-order thinking skills test. 

On the other hand, the gender results for the national sample seem at variance 
with the findings of other research (e.g., Noble & McNabb, 1989), and of the all- 
sophomore sample, in terms of the reductions in mean differences between gender 
groups when courses taken and course grades were statistically controlled. The reasons 
for this discrepancy are not obvious, but a reasonable hypothesis involves the nature of 
the samples. Noble and McNabb (1989) studied student performance on the ACT 
Assessment; their sample consisted of high school juniors and seniors who had chosen 
to take the ACT Assessment tests (i.e., they had "self-selected"). The samples for this 
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study consisted of high school sophomores who took the PLAN test, not of their own 
choosing, but as an "in school" test. The all-sophomore sample also appeared more 
academically-able than the national sample. 

It seems reasonable that the sample for the ACT Assessment study and the all- 
sophomore sample would show less variability in terms of prior scholastic achievement, 
as measured by course work, than the national sample. These two groups tended to be 
more academically-able than the national sample. Higher percentages of sophomores 
in the all-sophomore sample than the national sample had taken the mathematics course 
work; mathematics course work taken also accounted for a greater amount of the 
variance in the PLAN scores of the all-sophomore sample than of the national sample. 
Consequently, controlling for indices of prior scholastic achievement would have less 
potential to reduce mean differences for the ACT Assessment study and for the all- 
sophomore sample, precisely because there were fewer differences in prior scholastic 
achievement to control for in the first place. 

Additionally, Noble and McNabb (1989), unlike both the present study and other 
previous research (e.g., Noble et al., 1992), did not control for educational needs and 
plans or high school attended. The differences in the findings of the studies could also 
be linked to this difference in method. 

Though mean score differences by ethnic group were substantially reduced by 
controlling for course work taken, educational needs and plans, and high school 
attended, mean score differences remained. These findings, however, should not lead one 
to conclude that Caucasian-American students are more predisposed than African- 
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American or Hispanic students to acquire higher-order thinking skills, or that using the 
PLAN tests would disadvantage students from certain ethnic groups. Ethnicity 
accounted for only 2% of the total variance in PLAN scores, over and above courses 
taken, educational needs and plans, and high school attended. 

Planned course work in foreign languages was consistently negatively associated 
with PLAN scores. The simple correlations were not statistically significant, however. 
One possible hypothesis concerning this finding is the increasing numbers of schools 
requiring foreign languages of all of their students, regardless of their academic ability. 
A complementary hypothesis is that the more academically-able students completed or 
were currently fulfilling their foreign language requirements, where the less 
academically-able students postponed their language course work until later in high 
school (i.e., were still planning to take the course work). Further research is needed on 
this issue to support these hypotheses, however. 

High school attended appears to be a key factor in how students acquire higher- 
order thinking skills. The results of this study indicate that though students from 
different high schools may take similar courses, they may not be learning the same skills 
and knowledge; including course grades in future studies would help solidify this 
conclusion. Further, effective skills acquisition appears positively related to the financial 
status of the school and of students' families in the school districts. However, the 
relationship is not perfect; some lower SES schools appear to be effectively helping 
students acquire these skills. Future research on factors related to quality of education 
and noncognitive factors such as student's family income, parent's level of education, 
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the amount of time spent studying, and teacher and student motivation and support 
would be helpful in addressing these issues. The use of these variables in future 
research could well reduce the amount of unexplained variance and substantially 
decrease the contribution of ethnicity to an explanatory model. 




4b 



35 



REFERENCES 

Noble, J. (1994). PLAN National School Profile Summary Report. American College 
Testing. 

Noble, J. (1995). The PLAN technical manual (third edition). American College Testing. 

Belsley, Kuh and Welsch (1980). Regression Diagnostics. New York: Wiley. 

Chambers, G.A. (1988). All of America's children: Variants in ACT test scores - what 
principals need to know. [A paper presented at the annual meeting of the National 
Association of Secondary School Principals.] 

Noble, J. (1990). Using P-ACT+ in the Tennessee comprehensive assessment program. 
Unpublished manuscript. 

Noble, J. (1991). Plan as an every-student test. Unpublished manuscript. 

Noble, J., Crouse, J., Sawyer, R. and Gillespie, M. (1992). Ethnic/Gender bias and the 
differential preparation hypothesis: Implications for performance on the ACT Assessment . 
[A paper presented at the annual meeting of the American Educational Research 
Association.] 

Noble, }. and McNabb, T. (1989). Differential course work in high school: Implications for 
performance on the ACT Assessment. (ACT Research Report No. 89-5). Iowa City, IA: 
American College Testing. 

Pallas, A.M. and Alexander, K.L. (1983). Sex differences in quantitative SAT 
performance - New evidence on the differential course work hypothesis. American 
Education Research Journal, 20, 165-182. 

Reynolds, C.R. and Brown, R.T. (1984) Perspectives on bias in mental testing. New York: 
Plenum Press. 




4 



APPENDIX A 

Descriptions of Clusters for Planned Course Work 
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Cluster Subject Area Courses included in cluster 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 



English 

English 

English 

Math 

Math 

Math 

Math 

Soc. Studies 
Soc. Studies 
Soc. Studies 
Soc. Studies 
Soc. Studies 
Soc. Studies 
Science 
Science 
Science 

Foreign Language 
Foreign Language 
Fine Arts 
Fine Arts 
Fine Arts 



English 11, English 12 

English 11, Speech 

English 11, English 12, Speech 

Algebra 2, Trigonometry 

Geometry, Trigonometry 

Algebra 2, Geometry, Trigonometry 

Algebra 2, Geometry, Trigonometry, Calculus 

World History, American Gvt. 

World History, Geography 

World History, American Gvt., Economics 

World History, American Gvt., Economics, Geography 

American Gvt., Economics 

American Gvt., Economics, Geography 

Biology, Chemistry 

Biology, Chemistry, Physics 

Chemistry, Physics 

Spanish 

Spanish, French 

Art 

Music 

Art, Music 
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Descriptive. Statistics and Correlations for Independent Variables 
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Course work cluster 
taken/taking 


1 Eng. 9 or 10 


2 Eng. 9 & 10, or Eng. 

9 & Speech 


3 Alg. 1 & 2, or Alg. 1 

& Geometry 


4 Alg. 1 or Alg. 2 or 

Geometry 


5 Art or Music or 

Drama 


6 Any Foreign 

Language 


7 Foundational course 

•work 


8 2 yrs. Eng., 2 yrs. 

Math.,1 yr. Soc. 
Studies, 2 yrs. Sci. 
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Means and Standard Deviations for Need Variables, and Correlations with PLAN Scores 
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Percentages of Students Planning to Take Course Work Clusters, 
and Correlations of Clusters with PLAN Scores, by Sample. 
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